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[57] 



ABSTRACT 



A physically realizable one terabit or more ATM packet 
switch that has a large number of input interfaces connected 
to a single stage switching fabric which is in turn connected 
to a number of output modules, generally according to the 
growablc packet switch architecture. This ATM packet 
switch is different from other growable packet switches in 
that it has a single stage switch fabric controlled by an 
out-of-band controller, yet it has significantly reduced com- 
plexity with respect to comparably sized electronic crossbar 
switches or their isomorphs. The out-of-band controller has 
multiple priority levels in order to provide high priority 
users with a near certainty that their packets will be suc- 
cessfully routed, while delivering an acceptably low packet 
or cell loss probability to users at the lowest priority leveL 

21 Claims, 17 Drawing Sheets 
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R'uNOPj-l x B 2,i 




i.7 


SR H ABCD,i. x ..?ojl : 5: 


R 'ABC0,i x B 1.i 


R "uM0P ( M x B 2,i 




i,8 




R "ABCD.i x B 1.i 


R, ABC0,i x B 2J 


* 3 


i.9 




S R !EFGKi.. x »5i',i:S§: 


RM ABC0.i x B 2.i 


R 'ABCD,i x B 3,i 


i.10 




?h?..efgh.L x . B i.<; : : : : : >: : 


R EFGH,i x B ii 


R,, ABC0,i x B 3,i 




* 0 




R "EFGH,i X B 2.i 


R 'EFGH,i x B 3,i 


i+1,0 


R, EFGH,i x B 0,i+1 




:->:-v>>: R UKU x 3ikm 


R, *EFSH f i x 0 3,i 


i+1.1 


R"EFGH.i X B 0,i+1 






R'uKUi x B 3,i 


i+1.2 


R'lJKU X B 0,i+1 


* 1 




R "UKU x B 3,i 


i+1,3 


R,, UKUi x B 0,i+1 


R 'UKLJ X B U+1 




W$MHQPj* B iiMi : : 


i+1,4 


R'MNOP.i x B 0j+1 


R ' UKL.I x B 1,i+1 




: : : : : : : : : : : R,, MNOP ( i x B 3,i: : 3 : : 


i+1,5 


R"MNOP,i x B 0,i+1 


R MN0P p i x B 1,i+1 


* 2 




i+1,6 


^ABCD,i+1^Jo ( i+1^ 


R ' UKOP,i x B 1,1+1 


R *MNOP.i x B 2j+1 




i+1,7 




R *ABCD,i+1 xB 1,i+1 


R "uNOP,i x B 2j+1 




i+1.8 




R'aBCOJ+I x B 1.i+1 


R, ABC0,i+1 X B 2,i+1 


* 3 


1+1.9 




>:v: R, EPGH,i4i . x B l,i+1>;:: 


RM ABCD f i+1 x B 2,i+1 


R ABCD,i+1 x B 3,t+1 


i+1,10 




M7EFCH^1. x . B Vj+B: 


R, EFGH.i+1 x B 2,i+1 


R,, ABCD,i-t-1 x B 3,i+1 


i+1.11 


* 0 




RM EFGH^+1 x B 2J+1 


R EFGH.t+1 x B 3,i+1 


i+2,0 


R'efgh^+i x Bo.H-2 




UKL,i+l x . B 2,i+i 


R EFCH^fl x B 3,i+1 
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TERABIT PER SECOND PACKET SWITCH The extremely rapid growth of packet switching traffic 

HAYING ASSIGNABLE MULTIPLE PACKET carrying voice, computer (LAN/WAN), facsimile, image 

LOSS PROBABILITIES 311(1 video data to an ever widening variety of locations, 

along with the proposals used as part of the National 

CROSS REFERENCES 5 Information Infrastructure, has challenged both the packet 

This application is related to the following co-pending switch arcm*erture S ^ protocols, 

applications: "A TERABIT PER SECOND PACKET Many vendors and service providers have joined forces to 

SWITCH" by Thomas Qoooan and Gaylord Richards, Fil- d f finc t a ^ obd s ^ dard limits f^ ct ^tehing ^; 

ing Date Dec. 30, 1994, now U.S. Pat. No. 5,544,160, ^ ices J^ C ^*f?*\* U ^ m ^ fa ^° n " ™ e ^ °J 

-TERABrr PER SECOND ATM PACKET SWITCH HAV to ^ coordinated effort has been toe rapjd^velopmcnt and 

ING DISTRIBUTED OUT-OF-BAND CONTROL" by deployment of an A^nchronous Transfer Mode (ATM) as a 

Thomas Qoooan and Gaylord Richards. Filing Date Dec. «*nted ?»» of ™TJ^ f^*"? 8 ^ 

30, 1994 now U.S. Pat No. 5.642349, 'TERABIT PER P a ^ *f * ave stochas^^s^ted^val rates 

SECOND PACKET SWITCH HAVING DISTRIBUTED acc ? din 8 l ° *° "f 5 * *™ . f^A 1 

OUT-OF-BAND CONTROL OF CIRCUIT AND PACKET 15 pa^t-onented stondard, but unlike most of* data packet 

SWITCHING COMMUNICATIONS'* by Thomas Cloonan Predccess^s £25, frame relay, etc* ATM short, 

and Gaylord Richards, Filing Date Dec. 30, 1994. now U.S. **** P"** ™ ^ ^ ^f 50 

Pat Na 5.537,403. 'METHOD AND APPARATUS FOR uses a ve^ strean^ned form of err* ^recovery and flow 

DETECTING AND PREVENTING THE COMMUNICA- contro! relative to i s r^ccssors to fact, the ATM standard 

TION OF BIT ERRORS ON A HIGH PERFORMANCE 20 csseittiaUy ehmmates most error protccUoo and flow control 

™ D^l^ON ™^ of the short cells with uuWl netwo* delay 

^mktTod 'for SSSSMFa ~ ^tZ^~ 

GROWABLEPACKBT SWITCH" by Thomas Cloonan and ^r^Z JL^^TL Srv ' 

Gaylord Richards Filing Date Dec. 30, 1994, now U.S. Pat and the pubUc-sw^ched networkisa reality. 

No 5,550,815. and 'TERABIT PER SECOND ATM As ^ lcvel of connectivity becomes available to Ae 

PACKET SWTTCH HAVING OUT-OF-BAND CONTROL 30 cnurc spectrum of users, the demand and volume of ATM 

WITH MULTICASTING" by Thomas Cloonan and Gaylord J»»f m * * generated in the future is virtually 

Richards, Filing Date Aug. 31, 1995; Ser. No. 08/521,676 1™*** As » rcsult > *». s ? e of * c 

currentlY pendine and cross-connects required to route the projected ATM 

packet traffic may also grow by phenomenal rates within the 

TECHNICAL FIELD 35 next decade. ATM switches and cross-connects fox toll and 

The invention relates to large telecommunication gateway applications may require aggregate bandwidths 
switches that use data packets in order to communicate at from 155 gigabits per second (1000 inputs at 
aggregate throughputs at the one terabit per second leveL SCOTT OC-3 155 Mbps rates) to 2 4 terabits per second 
and more particularly to such large telecommunication ( 100 ° m P uts * SONET CXM8 2 4 gigabits per second 
switches that provide a very low cell loss probability to 40 rates). Additionally, if demand for broadband services to the 
higher priority users and an acceptably low cell loss prob- homc ^ LAN/WAN connectivity through the public- 
ability to lower priority users. switched network grows as some experts believe, then local 

telephone exchange carriers may install ATM switches and 

DESCRIPTION OF THE PRIOR ART cross-connects for metropolitan area network (MAN) appli- 

Telecomraunications have long used digital switching to 45 cations having aggregate bandwidths ranging from 100 

encode, multiplex, transmit and decode audio frequencies in Gigabits per second to 775 Gigabits per second, 

order to carry the millions of telephone voice calls of the By necessity, most of the current architectural research 

world. Telecommunication switches for voice calls have and hardware/software development for ATM switches has 

grown to very large sizes to keep pace with demand. Most concentrated on switches with much smaller aggregate 

of the switching systems that route and control voice call so bandwidths which meet the more near-term needs of the 

traffic are called circuit switches, which means that for each marketplace. For example, most projections within the 

call a type of bi-directional circuit usually audio is set up LAN/WAN community require aggregate bandwidths rang- 

berweeo the calling party and the called party. The circuit ing from 150 Mbps to 12 Gigabits per second, and most of 

that is set up has the bandwidth and transport timing the published proposals within the telecommunications 

necessary to simulate a face-to-face conversation without 55 industry call for aggregate bandwidths ranging from 20 

distortion or time delays that are objectionable to the parties. Gigabits per second to 160 Gigabits per second. Most of 

An alternative to circuit switching is called packet switch- these ATM packet switching architectures do not scale up to 

ing. For packet switching, the calling party is responsible for larger sizes. Scaling up present ATM switching architectures 

converting the information into one or more packets. This produce systems mat are cost prohibitive, size prohibitive, 

information could be encoded voice, it could be encoded 60 and/or physically unrealizable because of limits of the 

computer fla t a . or it could be encoded video. A Dumber to underlying to technology. 

guide the packet to its destination, Le. the called party, is For example, very common designs for large, high- 
typicaliy included in a packet header. The packet switching throughput switches use a multi-stage interconnection net- 
network then has the task of routing each packet to its work containing multiple stages of switching nodes (node- 
respective destination without undue distortion or delay. The 65 stages) interconnected by stages of links (link- Stages) to 
called party usually has the equipment to receive the packets provide multiple paths between input ports and output ports, 
and decode the information back into an appropriate form. Qos. Banyan and Benes networks are examples of such 
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networks. A multiple stage network design can yield net- Besides the ATM cell losses because of internal 
works with very high levels of performance (low blocking contentions, in an ATM packet switch where all of the N 
probabilities, low delay, high degrees of fault tolerance, . cells arrive simultaneously at the inputs of the distribution 
etc.). and may result in low system-level costs, because network, the cells must be processed at each stage and path 
network resources (nodes and links) are time-shared by the 5 hunts must be processed before the next group of N cells 
many different paths that can be set up within the network. arrives at the network input ports. If for example, the 
Physically realizing a multistage network for a large incoming transmission lines support SONET OC-48 2.5 
throughput ATM packet switch is, however, a problem Gigabits per second bit-rates, then the group of N ATM cells 
becau sc of routing time and proccs sing power. that arrive together must be processed and sent on to the next 

Any large, high-throughput ATM switching architecture in stage of the pipeline every 176 nano seconds (the duration 
must address two fundamental issues that profoundly effect of an ATM cell on a 2.5 Gigabits per second link). For large 
the overall performance of the resulting ATM switch. The values of N, a substantial amount of processing power is 
first of these issues is cell loss due to blocking within the therefore required to complete the path hunt operations for 
internal links of the distribution network (alsotaown as the all N cells. (For an N=256, then at least 1.45x10* padi hunts 
switching fabric), and the second is cell loss due to conten- must be completed every second, which corresponds to an 
tion for output ports by two or more ATM cells that pass 15 average processing rate of one path hunt every 684 pico 
through the switch at the same moment in time. The first seconds). Present commercial microprocessors can process 
issue can usually be solved by designing a network with approximately 100 million instructions per second, If each 
sufficient switching fabric (nodes and links) such that raul- path hunt took only one instruction, these 1.45x10" path 
tiple paths exist between input ports and output ports. As a hunts would required the total processing power of at least 
result, if two or more ATM cells attempt to use the same 20 15 of microprocessors. Thus, a path bunt controller based on 
shared resource (nodes or links) within the switching fabric, something other than a single microprocessor will be nec- 
cach of the cells can usually find a path, thereby eliminating essary for a large ATM packet switch, 
most of the internal network blocking problem. The second Two approaches to solving the path hunting problem can 
issue requires some type of method and apparatus for be envisioned. One approach uses in-band, i.e. self-routing, 
handling cells simultaneously arriving at the same output 23 control techniques to perform the required path hunts. For 
pQjl in-band control techniques, the connection requests are 

A general technique for a switch to handle cells destined prepended to the ATM data cells and routed through the 
for the same output port is analyzed in an article, A Grow- switch along the same paths used by the following ATM 
able Packet Switch Architecture, trrr Transactions on payioad This approach typically requires processing ele- 
Communications. February. 1992. by Eng ct aL and in 30 ments to be distributed throughout all of the nodes in the 
another article The Knockout Switch, ISS AT&T Technical network. This results in relatively complicated hardware 
Papers 1987 by Yeh et aL This general design technique, as within each node of the network in order to perform local- 
shown in FIG. 1, segments a packet switch into two distinct ized path hunting operations (on only the cells that pass 
parts. An Nx(FN) distribution network (which provides for through that node) when determining how to route the 
N input ports) and a bank of K mxn output packet modules 35 arriving connection requests and ATM cells. The second 
(which provide for a total of M=Kn output ports). Given that approach uses out-of-band control techniques whereby the 
each of the links emanating from the distribution network is controller and switch fabric are logically separated with this 
required to be terminated at one of the inputs to one of the architecture. Connection requests are routed to the path 
output packet modules, it can be seen that the equation hunting controller before the control signals resulting from 
FN-Km must be satisfied. In the switch shown in FIG. 1. the <o the path hunt are injected into the switch fabric to set the 
switching fabric is a memory-less Nx(FN) fanout switch paths. This second approach requires that the out-of-band 
whose function is to route an arriving ATM cell to any of the controller have tremendous processing power, (as men- 
m inputs on the output packet module connected to the cell's tioned above), because of the many path hunt operations that 
desired output port. The output packet module is a mxn must be performed in a very short period of time, 
switch with buffers that are available for storing cells that 45 Since the path hunt operations in switches mat use 
must be delayed when two or more cells contend for a in-band control techniques are only based on localized traffic 
particular output port If the arriving traffic is uniformly information and not on global informatioa with respect to all 
distributed across all output ports and if the buffers within of the switch traffic, the connections resulting from in-band 
the outout packet modules are sufficiently large, then the path hunting may not always be routed in optimal fashion, 
ratio mn can always be chosen large enough to force the cell 50 As a result systems using in-band control techniques often 
loss probability within the network to be below any desired require more switch fabric (stages and nodes) to provide the 
cell loss probability level. In fact, if the network size (N) is same operating characteristics as a less expensive switch 
large and if R represents the switch loading, men the cell loss based on out-of-band control techniques. In addition, out- 
probability of a network with mxn output packet modules as of-band control ATM switch architectures share many sum- 
shown by Eng et al is given by: " lanties with the partitioning of many existing telecommu- 
nication switching and cross-connect products that have 
F\ccl = centralized controllers, so the development of a system 

based on a function architecture should yield fewer design 
U f l — E {(nRM-ntyyV- 1 + MrM-n^- problems than an architecture based on newer architectural 

L t=o J 60 approach. Thus, an out-of-band control ATM switch should 

benefit from lower overall hardware costs and have more 
Existing small packet switches have acceptable cell loss standard architectural partitioning. On the other hand, the 
probabilities of approximately HT 12 , so any loss probability difficulties associated with performing path hunts in an 
smaller than existing units are considered acceptable. out-of-band controller for N arriving ATM cells and the time 
Although there are some users who would be willing to pay 65 required by a standard partitioned out-of-band controller to 
for an even smaller ATM cell loss probability if such were perform N path hunts tends to argue in favor of a newer 
available. in-band control switch architecture. For example assuming 



12/11/2003, EAST Version: 1.4.1 



5,724.352 

5 6 

a single path hunt requires at least one read from a busy-idle a telecommunication packet from its respective input inter- 
memory and one write to a busy-idle memory, N path hunts face to its desired output line. This path hunter hunts paths 
require 2N accesses to memory. If N=256, then the control- for telecommunication packets having at least two different 
lex must make 512 memory accesses every 176 nano levels of ATM cell loss priorities, 
seconds, so the average memory access time must be 340 5 

pico seconds. Since 340 pico seconds memories are not BRIEF DESCRIPTION OF THE DRAWINGS 

commercially available, a path hunt scheme different than fig. 1 is a block diagram of a generalized growable 

the present standard architectural portioning is required for _ a£ ^ ct 

any out-of band controller. ^\lL „ . ' , _„ , 
The high probability that large ATM switches will be 10 FIG. 2 is a slightly re-drawn FIG. 1. 
required coupled with the uncertainties and shortcomings of FIG. 3 is a block diagram of a growable packet switch in 
present ATM architectures demonstrate a strong need in the which the switch fabric is partitioned into L multiple pipes 
art for a packet switch architecture that will operate with according to the present invention, 
throughputs at the terabit per second levels and yet may be FIG. 4 is a block diagram, similar to FIG. 3, of a specific 
built using components manufactured by existing technolo- 15 embodiment of the present invention having four pipes 
gies. Also, the desire of some users for even lower ATM cell (L=4) and showing a configuration for the pipes, 
loss probabilities shows a need in the art to provide a large yiQ, 5 is a simplified block diagram of the embodiment 
high throughput ATM switch that has multiple ATM cell loss shown in FIG. 4 which shows greater details of the control- 
priorities, lex. 

It is an object of the present invention to provide an ATM 20 mQ 6 mustratcs ^ ^ sequences of requests to the 

packet switch architecture that has a large aggregate band- ^olla shown in FIG. 5. 

TL another object of the invention to provide a high *1G. ^ *t^ I !?' > ^ e< * Nock diagram of an embodiment of 

performance ATM packet switch that has a relatively low 10 0Ut P ut nMMUUc - 

sy stem-level cost 25 FIG. 8 is an illustrative example of rolling and its opera- 
It is another object of the invention to provide an ATM tion in a plan view of an amusement park and its satellite 
packet switch with a low cell loss probability with random parking lots. 

traffic, bursty traffic, and focused loads that is better than FIG. 9 shows plots of calculated values of various ATM 

present packet switches. cell loss probabilities both with and without the assignment 

It is another object of the invention to provide an ATM 30 0 f preferences, 

packet switch with multiple cell loss probabilities, but even pjQ 10 1$ a simplified block diagram of a representative 

the lowest cell loss probability is as good or better than those switch controller and its link controllers, 

of known ATM switches. piG. 11 is a detailed logic diagram of a link controller. 

SUMMARY OF THE INVENTION 3J FIG. 12 is a state table for the link controller shown in 

Briefly stated, in accordance with one aspect of the n ^ 

invention, the foregoing objects are achieved by providing FIGS. 13A-13D when joined together show the operation 

an ATM packet switch architecture having line cards with of a switch controller in response to a sequence of requests, 

one ATM cell period storage, a single stage, reduced inter- piQ, 14 illustrates the rolling of path hunting requests 

connects switch fabric, an out-of-band controller operating 40 through a switch having four pipe controllers with a single 

with multiple cell loss priorities, and output packet modules. i cve l of cell loss priority. 

In accordance with an other aspect of the invention, the pjQ 15 illustrates the rolling of path hunting requests 

foregoing objects are achieved by providing a packet switch through a switch having four pipe controllers with two levels 

for a switching telecommunication packet from any of a c f ce m 0 ss priorities. 

Dumber of input lines to any of a number of output lines. 45 

This packet switch includes a number of input interfaces, DETAILED DESCRIPTION 

each having an input port connected to a respective input ^ ^ , . . 

line of the number of input lines and an output port Each of Referring now to FIG. 2. a large, generalized switch 10 for 

the output ports is fanned out to F input ports, where F of a ATM communications, is shown in block diagram form, 

switching network, which has I input ports and P output 50 ATM switch 10 has a number of input interfaces U<r J W 

ports. F is the fan-out number, which is an integer greater * switch ^bnc 14, and bufferedoutput modules 16 0 -16^ 

than 1. lis a number that is an integer raultipl e of the number For ATM operation, input interfaces 1V1W high 

of input lines, and P is a number that is an integer multiple «P~* amplifiers that serve as a matching networks 

of the number I of the input ports. The switching network is ™* P° wcr amplifiers for fanning out iniornwaon received 

partitioned into a number. C of pipes, where C is an integer 55 on their inputs to multiple input ports of the switch fabnc 14. 

having a value equal to P divided by I. A number output Each of the input interfaces 1^-12^ also needs a capa- 

moduks. which together have a number of output module *>ility *> onc ATM cell, as will be explained below, 

inputs, are connected to P output ports of the switching Similarly for ATM operation, buffered output modules 

network. Each of the output module inputs is connected to are concentrators that are buffered to reduce 

a respective output port of the P output ports of the switching 60 packet loss when two or more packets are directed to and 

network. The output modules together have a number plu- contend for the same output of outputs OuVOuW 

rality of outputs, each of these output module outputs is Switch fabric 14 includes a fanout F where each of the 

connected to a respective output line of said number of outputs from the input interfaces 1^-12^ is fanned out to 

output lines. Each pipe of the C pipes has a path from each F inputs within switch fabric 14, such that if ATM switch 10 

of the number of inputs lines that is connectable to a 65 is an NxN switch then switch fabric 14 will have FN internal 

respective output line of the plurality of output lines. Also, inputs and FN outputs to output modules 16^-16^. Output 

the packet switch includes path hunter for hunting a path for Modules 1(S 0 -16 V _ 1 have a fanin or concentration factor of 
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F in order to convert the FN outputs of the switch fabric 14 ATM switch 10A also has a controller 20 which has the 

to N output module outputs Out 0 -Out N ^ 1 . Each output tasks of hunting and finding an available pipe through the 

module 16o-16 v _, stores arriving ATM packets in FIFO switch fabric 14A for each ATM packet. The controller 2# 

queues, and theo routes the ATM packets at the front of each uses the fact that the switch fabric 14A is partitioned into 

of these FIFO queues to their desired outputs Outo-Out^ 5 four pipes to break fee pipe hunting tasks into four parallel 

when the output ports are available. pipe hunting tasks that are each temporally shifted by an 

Switch fabric 14 is a general distribution network which acceptable amount Details of one embodiment of such a 

may be a network of switches, specifically crossbar controller 20 are shown in FIG. 5. 

switches, to provide multiple paths from each of its input Far the 0.640 Terabits per second, N=256 embodiment 

ports 170-17^1 to each of its output ports 19 0 -19 FN _ V lQ mentioned previously and shown in FIGS, 4 and 5. the 

However, it becomes highly impractical to make an NxN controller 20 may be contained on approximately eight 

switch out of a single crossbar to operate as the switching printed circuit boards. Controller 20 would accept up to 256 

component of switch fabric 14 when the size of N exceeds sixteen-bit request vectors from up to 256 line input inter- 

32. Thus, some other way is needed to realize the general faces IIq-H^} and perform path hunts on each of these 

architecture shown in FIG. 2. 15 request vectors within each 176 nanosecond. ATM cell 

Referring now to FIG. 3, an ATM switch 10A that is both interval to create the 1024 sixteen-bit connect vectors used 

practical and possible for N inputs where the size of N is at to established connections within the switch fabric 14A. 

least 256, is shown. Multiple paths from each input This requires that controller 20 operate with a processor 

17 0 _17 M _ 1 through the switch fabric 14A are provided to clock rate of at least 46 Mbps. This moderate clock rate 

prevent blocking. These multiple paths are partitioned into M permits the logic within the controller 20 to be implemented 

groups called pipes with each pipe providing exactly one with off-the-shelf CMOS EPLD's or similar devices, thus 

path between each input port 170-17^ and each output making the cost of the controller 20 (in large quantities) very 

port Wq-IV^ within the network. Thus, switch fabric reasonable. 

14 A is made up of multiple pipes ISq-IS^. The output The movement of request vectors from the input inter- 
modules 1^,-16^.! are essentially the same as the output M faces IIq-IZ^ to the controller 20 and the movement of 
modules shown in FIG. 2. connect vectors from the controller 20 to the crossbar 

Switch fabric as seen in co-pending and commonly switches 15 0 -15 & of the switch fabric 14A is a challenging 

assigned application entitled TERABIT PER SECOND task, because large amounts of control information must be 

DISTRIBUTION NETWORK, which is hereby incorpo- transported every 176 nano seconds ATM cell interval. For 

rated by reference, is a single stage, memoryless, and 30 example, in an ATM switch containing 256 input interfaces, 

non-self routing network. Since the switch fabric 14 A is not 256 16-bit request vectors must be transported to the con- 

unconditionaUy non-blocking as a full NxN crossbar switch trailer 20 every 176 nano seconds, leading to an aggregate 

would be. a controller 20 is included to hunt for a path bandwidth of 23 Gigabits per second between the input 

through the four pipes for each ATM cell. Since each of the interfaces sub-system and the controller 20 sub- system. In 

pipes 180-183 contains a path that could transport the ATM 35 addition, 1024 16-bit connect vectors must be transported to 

cell, the real purpose of the controller 20 is to find a path that the switch fabric 14A every 176 nano seconds to control the 

is not blocked crossbars switches 15 0 -15 63 . This requires an aggregate 

For ATM switch 10A, if the number of input lines. N is bandwidth of 93 Gigabits per second between the controller 

equal to 256 and if each input line is operated at a standard 20 sub-system and the switch fabric 14A sub-system. This 

2.5 Gigabits per second data rate, its aggregate throughput 40 93 Gigabits per second connect vector information can be 

will be 0. 640 terabits per second. Scaling or growing such an compressed into 29 Gigabits per second (given that only one 

ATM switch by a factor of two to 5 12 input lines and output input can be routed to an output during each ATM cell 

lines should be straightforward and result in aggregate interval) by standard compression techniques. However, 

throughputs of greater than 1 Terabits per second. Scaling to since this control information should be delivered with high 

an ATM switch size of 1024x1024 is considered within the 45 reliability, all of the control connections or control links 

present technology, and the architecture of the present between these sub- systems should be dually redundant (not 

invention is believed to be extensible even further as the shown in FIG. 4), so mere is actually 46 Gigabits per second 

speed of commercially available components increases and of data moving between the input interfaces cards and the 

as new. faster technologies are developed. controller 20 and 58 Gigabits per second of data moving 

Referring now to FIG. 4. a specific enibodiment of an 50 between the controller 20 and the switch fabric 14A. 
ATM switch 10A is shown. In this specific embodiment Preferably, high-speed serial links 22 will be used to trans- 
ATM switch 10A has two hundred fifty six input interfaces mit this control information. For such a case, input interfaces 
120-12233 which are connected to two hundred fifty-six 12o-12a5 would be grouped by fours such that only sixty- 
ATM input lines Lv-In^. The outputs of the input inter- four serial links would be required to move request vectors 
faces are connected to the input ports 17 0 -17j^j of the 55 from the input interfaces 12^-12^ to the controller 20. and 
switch fabric 14A The switch fabric 14A contains a total of 128 serial links would be required to move the resulting 
sixty-four 16x16 crossbar switches ISq-IS^ which are connect vectors from the controller 20 to the pipes 180-183 
partitioned into four pipes 18^18,. The fanout F is equal to (assuming the aforementioned data compression techniques 
four which if the number of output ports=FN results in 1024 are applied to the connect vectors), 
output ports 190-191023. The output ports 19 0 -19 l023 are 60 While the use of out-of-band control techniques does 
respectively connected to the inputs of sixteen 64x16 output require the additional hardware cost of these high-speed 
packet modules 16 0 -16 13 . The sixteen 64x16 output packet serial control links 22, these links 22 cause very little 
modules are connected to two hundred fifty six outputs increase the overall system hardware cost Considering that 
Outo-Out^jj. Those skilled in the art will recognize that the 256-input ATM switch lOAof FIGS. 4 and 5 already has 
other combioatioos of components could have been used, for 65 1024 high-speed serial links required to route ATM cells 
example thirty two 32x8 output modules could have been between the input interfaces 120-12233, and the switch 
used instead of the 64x16 output modules shown in FIG. 4. fabric 14A (when the fanout of four is included) and 1024 
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more high-speed serial links are used to route ATM cells and final path hunt attempt provides a very high probability 

from the switch fabric outputs 19 0 -19 1023 to the output of an ATM cell successfully being routed. Goal (5), by 

packet modules lfio-16 13 . Thus, the addition of the 192 packing many ATM cells in one portion of the network, 

serial links 22 for routing of the control information superficially seems to conflict with goal (1) that requires the 

increases the total number of high-speed serial links within 5 traffic be spatially distributed across the network. However, 

the system by merely nine percent. as will be explained below, temporal spreading provided by 

Applying the calculations of Yen et al. from the article the rolling technique permits the network to simultaneously 

'The Knockout Switch" the ATM cell loss probability of the satisfy both goals (1) and (5). . 

ATM switch 10 A shown in FIGS. 4 and 5 is 4.34xl0" 3 , Assuming that each of the 256 input ports 170-17^ of 

assuming that the connections of the inputs is symmetrical 10 FIG. 4 has an ATM cell mat needs to be routed through the 

and not independent as set forth in our co-pending applica- distribution network, and assuming that the switch fabric 

tion entitled 'TERABIT PER SECOND DISTRIBUTION 14Ais composed of four pipes 180-183, mcn out-of-band 

NETWORK". This cell loss probability falls short of the controller 20 may be required to perform 256x4=1024 

acceptable ATM cell loss probability of less than lxl(T 12 unique path hunts for the ATM cells before the cells can be 

mentioned previously. 15 routed. To distribute the ATM cells evenly across all four 

To reduce the ATM cell loss probabilities, controller 20 pipes, the 256 ATM cells requesting connections, the rolling 

applies a temporal spreading technique known as rolling, technique divides the requests into four groups of equal size, 

which provides many statistical advantages. Rolling The first group will have path hunts performed for its ATM 
involves and fulfills three fundamental goals that are aimed- cells in pipe 18q first, then in pipe 18 t . men in pipe 18 2 . and 

at providing more evenly distributed traffic loads. These 20 finally in pipe 18 3 . The second group wfll have path hunts 

goals are: (1) spatially distribute the traffic evenly across all performed for its ATM cells in pipe 18 t first, then in pipe 

pipes 180-183 so that one pipe will only carry its proper- 18 2 , then in pipe 18 3! and finally in pipe 18q. The third group 

tional fraction of the traffic load, (2) spatially distribute the will have path hunts performed for its ATM cells in pipe 18 2 

traffic evenly across all of the 16x16 crossbar switches first, then in pipe I83, then in pipe 18 0 , and finally in pipe 

15 0 -15 H within each pipe 18 0 -18 3 so that each of the 25 18,. The fourth group will have path hunts performed for its 

crossbar switches is equally loaded, and (3) temporally ATM cells in pipe 18, first, then in pipe 18^ then in pipe 18^ 

distribute the traffic that arrives in a given ATM cell period and finally in pipe lfcj. This ring-like ordering of the path 

across two ATM cell periods so that the traffic load can be hunts guarantees mat the routed ATM cells are distributed 

effectively decreased in an occasional ATM cell period when evenly across all four pipes. In addition, if the ATM cells 

an unusually high volume of traffic exists and is destined for 30 within each of the four equally sized groups are selected 

a particular output packet module. This effective lowering of such that the ATM cells within a single group can be routed 

the traffic load is accomplished by delaying some of the into exactly four of the 16 inputs on any 16x16 crossbar 

ATM cells arriving during a congested ATM cell interval. switch, then the routed ATM cells will also be evenly 

The cells arc delayed until the next consecutive ATM cell distributed across all of the 16x16 crossbar switches, 

interval when the traffic load competing for the popular 35 Referring now to FIGS. 5 and 6, a timing diagram for a 

resources, i.e. connections to popular output packet rolling technique according to the present invention is 

modules, will most likely be lower, so the delayed cells described To satisfy goals (1), (2), and (5) simultaneously, 

should have a higher probability of being routed in the next the out-of-band controller 20 uses the time delay/time dis- 

ATM cell interval. Since the switch fabric 14 A is tributxon described in goal (3), and these ATM cell delays 

mcmoryless. the ATM cells that must wait for the next ATM 40 required by goal (3) must be provided during each ATM cell 

cell interval are stored in their respective input interfaces interval. In all cases, when a group of ATM cells is passed 

120-122,,. around the ring-like structure of controller 20 from pipe 18 3 

In addition to satisfying these three fundamental goals of to pipe 18 0 , the controller 20 re-assigns the cells to the next 

packet traffic control to distribute the load, rolling also ATM cell interval (period) which requires that the ATM cells 

satisfies two further very important ATM system goals. First, 45 be delayed by one cell period. Because of this rc-assignmcnt 

goal (4) is that the ATM switch 10A must guarantee that and delay, each cell group encounters a very lightly-loaded 

ATM cell ordering can be simply maintained when an ATM set of 16x16 crossbar switches for its fourth and final path 

stream is re-constructed at an output packet module hunt An additional advantage of this rolling technique using 

16 0 -16 15 even if rolling causes some of the ATM cells re-assignment and delay of ATM cell intervals is that it also 

within the stream to be delayed differently than others. 50 allows more than 64 simultaneously arriving ATM cells to 

Secondly, goal (5) is that rolling must also guarantee that the be routed through the switch fabric 14A to any single output 

controller 20 will attempt to route every ATM cell through packet module 16^16^ (even though there are only 64 

each of the four paths to its desired output packet module, connections or links from the switch fabric 14A to each 

but each of the successive path hunt attempts must occur in output packet module 16o-16 15 ). This is occurs with the 

a more lightly- loaded 16x16 crossbar switch so that the first 55 rolling technique because all of the ATM cells do not need 

attempt occurs in a 16x16 crossbar switch with many to be routed during the same ATM cell interval. Thus, the 

previously-routed ATM cells (and very few available paths rolling technique when used in the out-of-band controller 20 

to output packet modules) while the fourth and final path results in extremely low cell loss probabilities both within 

hunt attempt occurs in an 16x16 crossbar switch that is the switch fabric 14A and the output modules lfio-16 13 . 

virtually empty (thereby providing many available paths to 60 even during a transient cell interval that has an extraordi- 

output packet modules). The rolling technique is similar to narily high traffic load. 

spatial path hunt techniques that pack as many calls as The one ATM cell period delays incurred by some of the 

possible in one portion of a spatial network, which by ATM cells as they are routed through the switch fabric 14A 

forcing near 100% occupancy in parts of a system results in would normally lead to the conclusion mat there would be 

the remainder of the calls having a very high probability of 65 difficulties in satisfying goal(4) of maintaining proper cell 

being successfully routed through the remainder of the ordering. However, the ring-like ordering of the path hunts 

system if usage is below 100%. Thus, rolling in its fourth within the out-of-band controller 20 guarantees that delayed 
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cells in a stream of ATM cells will always be routed through full However, if the customer does find his or her seat to be 

lower-numbered pipes than non-delayed cells (where pipe vacant on that tram, then the tram will deliver him or her 

18o is the lowest-numbered pipe and pipe 18a is the highest- straight to the amusement park 520. If the customer fails to 

numbered pipe). This information, coupled with the fact that get on the first tram, he or she must wait and try the second 
ATM cells are delayed by at most one cell period, ensures 5 tram which has already visited two other parking lot loading 

that proper cell ordering will be maintained if the cells are areas. If the customer is successful at rinding his or her 

extracted from the switch fabric 14A and loaded into first- pre- specified seat on the second tram, that tram will deliver 

in-first-out queues 1740-174,53 (shown in FIG. 7) of each ^e customer to the amusement park 520 after one more 

output module of the output modules 16 0 -16 13 in the order parking lot stop. If the customer fails to get on the first tram 
of the lowest numbered pipe to the highest numbered pipe: 10 ^ mc second tram, then he or she must wait and try the 

pipe 1%, pipe 18 ,, pipe Iftj. and pipe 183. third tram which has only visited one other parking lot 

Referring now to FIG. 7. the output module 1^> (and the loading area. If the customer is successful at finding his or 

fifteen other output modules 16j-16 15 ) may be a 64x16 her seat on the third tram, that tram will deliver him or her 

embodiment of the concentrator described in U.S. Pat No. t o the amuse meat park 520 after two additional parking lot 
5.412.646. entitled "ASYNCHRONOUS TRANSFER 15 sto ps. If the customer fails to get on any of the first three 

MODE SWITCH ARCHITECTURE", filed May 13, 1994, trams, then the customer must wait and try the fourth and 

by Cyr et al. and commonly assigned to the assignee of the fi na i tram. Fortunately, this tram has not visited any parking 

present invention, which is hereby incorporated by refer- lots yet, so the arriving tram Is empty, and the customer's 

ence. The output module 16q in FIG. 7 is a specific case of scat wQl be taken only if another customer in his/her parking 
the generalized concentrator shown in FIG. 4 of the above- 20 lot line Is also trying for the same seat. The system 530 

referenced patent application of Cyr et al. Since the output satisfies goal (5), because each of the successively arriving 

modules 16o-16 13 are well described in the above refer- trams is more lightly-loaded than the previous one. Thus, a 

enced application, in the interest of brevity they will not be controller 20 rolling ATM cells indeed can fulfill goals (1), 

further described here. (2), and (5). 

To provide a better understanding the equation of the 25 The rolling technique if used by itself improves the ATM 

rolling technique, a real-life analogy will be described with ^ loss probability of ATM switch 10A from 4.34xl0" 3 to 

respect to FIG. 8, which is a plan view of an amusement park approximately 1(T u . Using the analysis techniques of the 

system 500. Consider the problem of transporting a large article "A Growable Packet Switch Architecture" me cell 

number of people from amusement park parking lots 511, j oss probabilities for an ATM switch 10A that has indepen- 

512, 513, or 514 to the amusement park 520 using trams to 30 connections to inputs of the switch fabric 14 according 

shuttle the people between the two points. Tram system 530 t0 Galois field theory and also has an out-of-band controller 

is composed of four tram shuttle trains each with a prede- that incorporates rolling techniques can be analytically 

tennined route, which is analogous to the four pipes of modeled and calculated. Each of the 16x16 crossbar 

switch fabric 14A. Each tram shuttle train contains sixteen switches in pipe 18q receives an offered traffic load equal to 

cars (representing the 16x16 crossbar switches within a 33 Ra=R^/4+Rres, where Rres is defined to be the fraction of 

particular pipe), and each shuttle car is equipped with the 16 inputs to a 16x16 aossbar switch that are blocked in 

sixteen seats (representing the output links emanating from jg 3 and routed to pipe 18o for a re-attempt For a first 

a single 16x16 crossbar switch). In this analogy, each attempt at solving for the cell loss probability, let us assume 

customer (representing an ATM cell) arrives in one of four ftat rxz S =R l /16. Thus, the cell loss probability of a single 

parking lots 511, 512, 513, or 514 surrounding the amuse- 40 16x16 crossbar switch in pipe 18q can be determined using 

ment park 520. As a result each customer is instantly placed ^ equation of Eng et aL 
in one of four groups, and since the parking lots 511-514 are 

the same size, each group contains an equal number of F(ceUk*s) = 
customers on the average. The customers in any single 

parking lot 511. 512. 513. or 514 must then divide up and 45 [i-m^/fc)] [ 1 - 1 {{*Xtf<-*Ri)VU 1 +(*J?i)"*<-^«<: 
stand in one of sixteen lines, where each line is associated L fc*> J 
with a respective car of the tram shuttle train. The amuse- 
ment park 520 is sub-divided into sixteen different theme where m=l, n=l. and the switch loading is given by R^=RjJ 
areas (The Past Land. The Future Land, etc.), and each of the 4+RJ16. Using these assignments, the resulting cell loss 
sixteen seats of a particular tram car is labeled with the 50 probability for a fully-loaded (R^l.0) pipe 18 0 16x16 
theme area to which that seat's occupant will be given crossbar switch can be calculated to be: 
admission. Before arriving in the parking lot each customer 
must randomly chose one of the sixteen theme areas 

(representing the sixteen output packet modules 16o-16 15 ) P(fi*K k«s m pipe i« 0 )=ijxio- 1 . 

where he or she wishes to spend the day. Customers must 55 ... , ^ cc . t . t 

then find an available seat Associated with their desired Thus, the fraction of the 16 inputs to a 16x16 ^ssb^at 

theme area on one of the four trams that passes by the are passed to the second pipe after the first attempt is given 

loading area 531, 532. 533, or 534 of their parking lot If a by: 
customer has not found an available seat after four trains 

have passed by, then he or she is not permitted to enter the so /i-2=&xP(ceii k** in pipe M 0 M3.i3xi<r 1 >(i 3xic^)=4D6x 

amusement park during that day (This harsh condition |Q-a t 
represents the loss of an ATM cell due to blocking in all four 

pipes of the distribution network, a small but finite By symmetry, this should have also been the same as the 

possibility). fraction of inputs that are passed from pipe 18 3 to pipe 18^ 

The first tram that stops at the loading area that the 65 so the residue assumption of 1*^/16=0.062 above was incor- 

oi stoma- can try has already visited three other parking lot red. By refining this assumption and performing a second 

loading areas, so the customer's pre-specifled seat may be attempt and now assuming that Rres=R L /32, Thus, the cell 
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loss probability of a single 16x16 crossbar switch in pipe 18^ 
can be determined again using the equation of Eng et aL, 
where m=l, o=l, and the switch loading is given by Ra=R i / 
4+R i /32. Using these assignments, the resulting cell loss 
probability for a fully-loaded (Rj=1.0) pipe 18o 16x16 
aossbar switch is calculated to be: 

P(ceH kss in pipe U&lJxUT*. 

Thus, the fraction of the 16 inputs to a 16x16 crossbar thai 
are passed to the second pipe after the first attempt is given 

by: 

/i-2=&xP(ceU loss m pipe l« 0 )=(2-81xl0 ri Xl-2xl(r 1 )=3.37x 

This calculation result is very close to the assumed value 
of Rres=R^/32=3.13xlO -2 . so the assumption is considered 
to be satisfactory. The blocked cells are sent to pipe 18j for 
subsequent path hunting, and they encounter a negligible 
number of ATM cells from previous attempts. Thus, the 
16x16 crossbar switch in pipe 18 x can be modeled for 
analysis as a growable packet switch, with m=L, n=l, and 
R3=fl-2. and the resulting cell loss probability of this model 
is 1.4xl0~ 21 . The traction of the 16 inputs to the 16x16 
crossbar in pipe 18 t that are passed to the pipe 18 2 is 
4.2xl0~*. Similar arguments can be used to show that the 
resulting cell loss probability for cells entering pipe 18 2 is 
1.9xlCT\ and the resulting fraction of the 16 inputs to a 
16x16 crossbar passed to pipe 18 3 is 7.9x1 0~*. The resulting 
ATM cell loss probability in pipe 18 3 is 3.7x1 (T 6 , and the 
fraction of the 16 inputs to a 16x16 crossbar not routed in 
pipe 18 3 (and therefore not routed in all four pipe attempts) 
is 2.9x10"". Thus, through the use of the rolling techniques 
within the out-of-band controller 20, the ATM cell loss 
probability of an ATM switch 10A with independent con- 
nections at the inputs of its switch fabric 14A can be 
decreased from an unacceptable value of 1.47X10" 6 to an 
acceptable value of 2.9xl0~ 15 . 

A preference technique may be used in conjunction with 
the rolling technique described above to decrease the cell 
loss probability of an ATM switch 10A even further. Refer- 
ring back to FIG. 8 and the amusement park analogy, some 
form of arbitration was required at the tram loading areas to 
determine which of the customers in the line will be given 
a particular seat on the tram when more than one customer 
is requesting the same seat Similarly, the out-of-band con- 
troller 20 must provide an arbitration scheme for selecting 
which of the arriving ATM cells will be assigned a particular 
link whenever two or more cells request access to the same 
link. The arbitration scheme used can have an advantageous 
effect on the ATM cell loss probabilities. 

One possible arbitration scheme is a random scheme to 
determine which of the ATM cells is assigned the link. The 
random selection scheme is the scheme assumed for the 
analysis of the rolling technique presented above. However, 
other arbitration schemes are possible, and one particular 
arbitration scheme that has advantageous results is called the 
preference scheme. The preference arbitration scheme 
assigns a preference weight to each of the ATM cells in a 
particular grouping. ATM cells with higher preference 
weights are given precedence over ATM cells with lower 
preference weights whenever two or more cells request 
access to the same link. As a result an effective hierarchy is 
created within the groupings of ATM cells. 

The creation of a hierarchy may superficially seem to 
produce undesirable characteristics within the switch fabric 
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14A, because customers with high preference weights will 
be offered better service than customers with low preference 
weights. In fact, the one customer with the highest prefer- 
ence weight within each group can never have his or her 

5 ATM cell blocked by another customer's ATM cell. 
Although this may seem unfair, a detailed analysis of the 
effects of imposing this hierarchy indicates that it actually 
leads to improved performance, i.e, lower cell loss 
probabilities, for all customers — even for the customer at the 

10 bottom of the hierarchy with the very lowest preference 
weight 

The results of this analysis are summarized in FIG. 9, 
where the probability of loss of an ATM cell; Le,, the 
probability of a cell not being assigned to an available path, 

is is shown as a function of the number of path hunts that were 
attempted in different pipes by the out-of-band controller 20. 
In mis analysis, it was assumed that the group sizes were 
four— i.e., up to four ATM cells could simultaneously com- 
pete for access to the same link. As a result, four different 

20 preference weights were assigned to create a hierarchy for 
the four input ports associated with each group. The pref- 
erence weight associated with a particular input port is 
assumed to be a fixed constant that does not vary with time. 
The resulting plots 901. 902, 903 and 904 in FIG. 9 indicate 

25 that the cell loss probability decreases as more path hunts in 
more pipes are performed, but it also shows that the inputs 
with the lower preference weights 903, 904 have higher cell 
loss probabilities than the inputs with higher preference 
weights 901, 902, as might be expected Super-imposed on 

30 these plots is a similar plot 910 which indicates the prob- 
ability of not being served when a random selection arbi- 
tration scheme is used instead of the hierarchy arbitration 
scheme. The surprising and unexpected results are that after 
path hunt attempts in four different pipes, the random 

35 selection arbitration scheme produces cell loss probabilities 
which are higher than 1he average of the cell loss probabili- 
ties for the hierarchy arbitration scheme. In fact, the plot 910 
of the random selection arbitration scheme shows an average 
cell loss probabilities for all of the input ports which are 

40 notably higher than the plots 903 and 904 which are the 
average cell loss probabilities for even the input ports with 
the lowest preference weights within the hierarchy arbitra- 
tion scheme. This phenomenon can be explained by the fact 
that after three sets of path hunts in three different pipes, the 

45 distribution of ATM cell requests entering the fourth pipe is 
very different depending on whether the random or prefer- 
ences arbitration scheme is used. In the random selection 
arbitration scheme, there is a small but equal probability that 
all of the ATM cells arc requesting a path. However, in the 

so hierarchy arbitration scheme, most of the ATM cells with 
higher preference weights will be requesting a path with a 
probability of practically zero, while the ATM cell with the 
lowest preference weight will be requesting a path with a 
sizable probability, because that particular ATM cell may 

55 have been denied access to links in all three of its previous 
path hunt attempts. However, a single request arriving with 
a high probability at the fourth and last path hunter in the 
controller will lead to more routed ATM cells than many 
requests arriving with low probability, because the single 

60 request can always be satisfied since contention for an 
output link will never occur. 

As a result, it seems apparent from the plots in FIG. 9 that 
by assigning preference weights to the input ports and by 
using a hierarchy arbitration method to resolve link conten- 

65 tion and mute paths in the out-of-band controller, the worst- 
case cell loss probability of the switch fabric 14A can be 
decreased from 2.9xl0~ 13 that was achieved by the intro- 
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eviction of the rolling technique to an even lower value of within FIG. 5 provides terminations for a respective set of 

2.4xl(T 16 . It is worth noting that input ports mat are sixty-four links emanating from the switch fabric 14A. Each 

assigned higher preference weights will encounter even output module 16^,-16 13 also provides two basic functions: 

lower cell loss probabilities as indicated in FIG. 9. it provides a small degree of space switching to route each 

Referring back to FIG. 5, in order to provide a physical 5 ATM cell arriving on one of the sixty-four inputs to the 

embodiment of the rolling and preference methods, the ATM desired one of the sixteen output ports, and it provides 

switch 10A is segmented in to four basic sub-systems. These buffering of ATM cells to handle the problems associated 

four sub-groups consist of the input interfaces Xl^-M^ with multiple packets that arc simultaneously destined for 

the output modules l6o-16 J3 , the switch fabric 14A, and the the same output Outo-Out^y 

out-of-band controller 20. 10 There are many ways for these two functions to be 
The input interfaces LZq-I^j within the network pro- implemented. The most straight-forward approach would 
vide the necessary interfaces between the incoming trans- probably construct a shared memory switch that could 
mission HwW and the links connected to the switch fabric perform sixty-four memory writes and sixteen memory 
14A and the out-of-band controller 20. As a result, the input reads within an ATM cell interval (176 nano seconds). The 
interfaces Hq-II^ss must provide a termination for the 15 memory could then be treated as sixteen disjoint linked lists 
input transmission line. For example, if the input transmis- (one for each output Outg-Outj^) along with a seventeenth 
sion line is a SONET link, then the input interface must linked list containing idle memory locations. Although 
provide for clock recovery, link error detection, SONET simple, this approach requires eighty memory accesses 
pointer processing and frame delineation, ATM cell every 176 nano seconds, so it would demand memories with 
extraction, and an elastic storage function to synchronize the 20 2.2 nano seconds access times. An alternate approach would 
arriving ATM cells to the system clock within the distribu- split each 64x16 output module 16 0 -16 1S into a 64x16 
tion network. The extracted ATM cells are then loaded into concentrator and a 16x16 shared memory switch. The con- 
a FIFO buffer of the input interface. The input interface must centrator would be a memory system that provides for 
also read ATM cells from the FIFO buffer and extract the sixty-four writes and sixteen reads every ATM cell interval. 
ATM header from the cell. The VPI/VCI field of each ATM 25 but the memory size could be small (and memory speeds 
header is then used as an address into a translation table could be fast) since the buffering required for output con- 
located on the input interface. The output of the translation tention problems is not provided in mis memory. In addition, 
table provides a new VPI/VCI field and the address of the the 64x16 concentrator could be implemented as a single 
output packet module to which the ATM cell is to be routed. linked list spread out across sixty-four distinct memory 
The new VWVCI field is written into the ATM cell as a 30 chips. As a result, each memory chip would require only one 
replacement for the old VPI/VCI field, while the output write and up to sixteen reads for every ATM cell interval 
module address is routed as a request vector to the out-of- The 16x16 shared memory switch only performs thirty-two 
band controller 20 for the controller fabric 14A. Since the memory accesses every ATM cell interval, so slower (and 
amount of processing tune required by the out-of-band larger) memories could be used, and the buffering for output 
controller 20 is a fixed value, the input interface simply 35 contention problems could be provided in this shared 
holds the ATM cell in a buffer until the out-of-band con- memory portion of the output module. Thus, this latter 
trailer 20 has completed its path hunt and has relayed the arrangement is the more practical alternative for an output 
results into the switch fabric 14 A. Once the switch fabric module. 

14A is loaded with the new switch settings to appropriately The switch fabric 14A is essentially a group of small 

route the ATM cell, the input interface can inject the ATM 40 circuit switches that provide the required connectivity 

cell into the switch fabric 14A and it will be automatically between the input interfaces and the output modules in 

routed through the switch fabric 14A to its desired output response to the control signals generated by the out-of-band 

module 16 0 -16 lt . It should be noted that each input inter- controller 20. In the embodiment of the ATM switch 10A 

face 120-1223, actually is provided with one link to each of shown in FIG. 5, the switch fabric 14A is composed of 

the four pipes 18^-18, of the switch fabric 14A. In addition, 45 sixty-four 16x16 crossbar switches, where disjoint groups of 

the use of rolling (i.e. temporal spreading) within the switch sixteen switches comprise a pipe. The four pipes are labeled 

fabric 14A may require a copy of the ATM cell to be injected pipe 18^ pipe 18 A , pipe 18 2 , and pipe 18 3 . and the sixteen 

into each of the four links during any one of two consecutive 16x16 crossbar switches within a given pipe are labeled 

ATM cell intervals. As a result, the timing within the input switch 0-15. The crossbar switches must be capable of 

interfaces Uq-Hz^ must be tightly coupled and synchro- so receiving the control signals generated by the out-of-band 

nized to the timing of the rest of the sub-systems within the controller 20 and must re configure all of the switch settings 

ATM switch 10A. during a guard-band interval between consecutive ATM 

Each of the two hundred fifty six input interfaces cells. Each 16x16 crossbar switch supports sixteen inputs 

\2 0 -l2 255 in FIG. 5 are numbered with an address ranging labeled input A through input P, and each 16x16 crossbar 

from 0 to 255. but each input interface is also assigned an 55 switch also supports sixteen outputs labeled output AA to 

alias address given by a letter between A and P. These alias output PP. It was noted above that each input interface 

addresses are used to identify which input port the input connects to a different 16x16 crossbar in each of the four 

interfaces will connect to within the switch fabric 14A, The pipes 180-183, but it should now be noted that an input 

actual set of four crossbar switches to which a particular interface that connects to input X in pipe 18 0 is required to 
input interface is connected is determined by the Galois field 60 be connected to input X in the other three pipes 18 x -18 3 as 

techniques that were described previously. These techniques well, where X is an element of the set {A3, . . . P}. The 
guarantee independence between all of the inputs on any actual connections between the input interfaces 120-12^5 

16x16 crossbar switch of any pipe. and the crossbar switches within the switch fabric 14A are 

Each of the sixteen output modules 16 0 -16 13 in FIG. 5 is determined using Galois field theory techniques that were 
labeled with addresses ranging from AA to PP. and each 65 referenced above. These techniques guarantee independence 
output module performs an important function within the between input ports for routing within switches in each pipe 
ATM switch 10A. Each of the output modules 16 0 -16 15 of the switch fabric 14A. FIG. 5 also illustrates that output 
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YY from each of the sixty-four cros soar switches is routed input interface 12q in FIG. 5 routes its request vector to pipe 
to one of the sixty-four inputs on the 64x 1 6 output module hunting controller 24 0 where it is poked into the pipe hunting 
labeled YY, where YY is an dement of the set {AAJJB, . . ring (ie. controller 20), and the rolling scheme requires (he 
■ J*P}. request vector to be looped through pipe hunt controller 24 L , 

The basic function of the out-of-hand controller 20 for the 5 pipe hunt controller 24 2 , aod pipe hunt controller 24 3 as it 
switch fabric 14Ais to determine through which of the four circulates around the ring. In general, each of the input 
pipes 185,-183 a particular ATM cell may be routed. Once the interfaces 1V 12 255 produces one request vector, and each 
out-of-band controller 20 has successfully determined a pipe request vector will contain a number of bits equal to the 
through which the ATM cell is to be routed without being Dumbcr of output modules within the system The request 
blocked^he task of setting up the path through the pipe is 1Q vcct ^ from a ^ c ^ ut mtaface m FIG. 5 is thus a 
simple, because by me de&mUon of a pipe, there wfll exist ^ ^ whcrc ^ bit of mc rcqucst vcctor 

only one path within the pipe between .the ^input port of Ae teen modulcs ^ ^ ceU 

am vine ATM cell and the desired output module. As a result. *™ . . a . , ^ . . 

d.eSimotalpaihhaniii.gtaskofVswitchingiietworkfa waaa • "P* f^l^^l T"!' » 
«*atian y lefecrtto the stapta task of pipe huntm* m the P°* » *■ Hh output module , ta bit , Mkik 

ATM switch 10A. is request vector will be set to a logic '1 and all other bits 

The out-of-band controller 20 still requires a large busy- within (he request vector wfll be set to a logic «W. When the 
idle table to identify the status of each of the intermediate controller 20 receives this particular request vector from the 
(FN) links between the 16x16 crossbar switches of the input interface, it can then identify that a path is required 
switch fabric 14A and (he output modules 16 0 -16 l5 as busy between the source input interface and the i-th output 
and unavailable or idle and available. However, this large 20 module. 

bu sy-idlc table may be sub-divided into many small busy- The entire sixteen-bit request vector from a input interface 
idle tables that the controller 20 can access in parallel, and is routed via a respective control connection 210-21255 to 
thereby perform many pipe hunting operations in parallel. one of the four pipe hunt controllers 24 0 -24 3 , and the 
There are many ways to implement the controller 20 for a controller 20 pokes the vector into one of the sixteen switch 
large switch having the general growable packet switch 25 controllers associated with that particular pipe hunt control- 
architecture- In the extreme case, four levels of parallelism ler. As shown in FIG. 10, the sixteen bits of the request 
may be applied to the architecture of the controller 20 to vector arc injected into a switch controller and are distrib- 
perform pipe hunting. One embodiment that uses three uted across all sixteen of the link controllers within that 
levels of parallelism will be described in detail, first and then particular switch controller. Each link controller is assocd- 
a fourth level of parallelism for the controller 20 will be 30 ated with a single link between the crossbar switches and the 
discussed. output modules, and it essentially processes one bit of the 

The first level of parallelism is obtained by providing each sixteen-bit request vector. This finite state machine circuitry 
of the four pipes 180-183 with a respective pipe hunt that is associated with* single link controller consists of one 
controller 24 0 -24 3 . This level of parallelism allows pipe flip-flop (the single-bit memory required to store the busy- 
hunting to be carried out in all four pipe hunt controllers 33 idle bit associated with this link controller's link) and four 
24q-24 3 simultaneously. The second level of parallelism is logic gates (shown in FIG. 11). A state table description of 
obtained by providing switch controllers 26 0 -26 w , with the link controller operation is given in FIG. 12, where the 
sixteen switch controllers within each pipe hunt controller state variable is defined by the busy-idle bit The link 
24 0 -24 3 . A unique switch controller 26o-26 63 is respectively controller hardware provides for one request vector input bit. 
associated with each of the 16x16 switches within each pipe 40 designated request-in; one request vector output bit, desig- 
of the switch fabric 14A. As a result, pipe hunting operations nated request-out; and one connection vector output bit 
can be carried out in parallel within all sixteen of the switch designated connect The request vector input bit is a logic 
controllers of each pipe hunt controller 240-24,. The third "1" if the input desires a connection through the link 
level of parallelism is obtained by permitting each of the associated with this link controller— otherwise, it is a logic 
switch controllers 2^0-26^ to perform parallel processing 45 "0". The request vector output bit is a logic "1" if the logic 
over all sixteen of the output Hnfcc attached to its respective "1" input request vector bit was not satisfied by this par- 
16x16 crossbar switch. Effectively, each of the switch con- ticular link controller— otherwise, it is a logic "0". The 
trailers 2€o-26 63 reads sixteen busy-idle bits from its busy- connect vector output bit is a logic "1" if the logic "V input 
idle memory in parallel, performs parallel pipe hunting request vector bit was satisfied by this particular link con- 
operations based on those sixteen bits, and then writes the 50 (roller indicating the ATM cell will be routed to its desired 
sixteen resulting busy-idle bits into its respective busy-idle output module through the link associated with this link 
memory in parallel with the other busy-idle memories. A controller— otherwise, it is a logic "0". The busy-idle flip- 
representative switch controller 26^ of the sixty four switch flop in FIG. 10 is reset to the logic "0" (idle) state at the 
controllers 26 0 -26 63 is shown in FIG. 10. The concurrent beginning of each ATM cell slot, so the first request vector 
processing of sixteen busy-idle bits is accomplished by 55 bit that enters the link controller with a logic "1" request is 
providing switch controller 26 0 sixteen unique link control- assigned the link (creating a logic " V connect vector bit and 
lers AA-FP. each of the link controllers AA-PP is assigned a logic "0" output request vector bit) and sets the busy-idle 
the task of processing busy-idle bits for one intermediate flip-flop to the logic "1" (busy) state. Any subsequent 
link between its portion of the switch fabric 14A and its request vector bits that enter the link controller during this 
respective output modules. In the embodiment shown in 60 particular ATM cell slot will be denied a connection through 
FIG. 10, the large busy-idle memory required to control this link (forcing a logic "0" output on the connect vector bit 
. switch 10A has been divided into many single bit memories, and creating an output request vector bit that is identical to 
busy-idle flip-flops, with each single bit. busy-idle memory the input request vector bit). A time-lapsed view of several 
being logically and physically associated with its respective consecutive sixteen-bit request vectors passing through a 
link controller AA-PP. 65 single switch controller is shown in FIG. 12, along with the 

The general data flow for request vectors generated by the resulting states of the busy-idle bits stored within the switch 
input interfaces XIq-M-^ is shown in FIG. 5. For example, controller. The resulting output request vectors and output 
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connect vectors illustrate the general operation of each of the rates, the aggregate throughput would be over 1.0 Terabits 

pipe hunt controllers 24^-24 3 . per second. For switches of that size, a fourth level of 

The use of rolling within the controller 20 requires a very parallelism may be needed to provide sufficient processing 

precise temporal ordering of two fundamental events: pok- Po wcr for the controller 20 to hunt for all the paths through 

ing and busy-idle flip-flop clearing. The timing diagram of 5 an*c pipes in real time. For ATM switches with 512 and 

FIG. 13 illustrates the synchronization and data flow mat 1024 input lines, the data rates on oonn^onswthin thcir 

might be used for the logic within the controller 20. As TOSJ ^ v ^^°i^^2^ ??^5^ 3 f ^^^^ ^ 

indicated by the timing diagram, the flow of data around the cons«lerably higher mat the 113 1Mbps rate of the 256 input 

ring of confer 20 is from pipe — er 24 to p^e ^™ ^ lcvd of paralleUsm is a 

^ntrolle^ to pipe co«troUer 24 2 to pipe controller 24, to modificatlon of ^ ^^ly Ascribed Controller 20 

and back to pipe controller 24* Request vectors generated dcsi which mat request vectors be routed through 

by input interfaces with alias addresses A, B, C. and D are ^160* hunter stages m parallel. In particular, all of the 

poked into pipe controller 24,. Request vectors generated by request vectors that are poked into a particular pipe are 

input interfaces with alias addresses E, F, G. and H are poked routed through the pipe hunter stages together, and these 

into pipe controller 24^ Request vectors generated by input is request vectors are said to comprise a poke group. In the 

interfaces with alias addresses L J, K, and L are poked into. embodiment shown in FIG. 5, mis approach to the design of 

pipe controller 243. Request vectors generated by input controller 20 creates four poke groups of sixtceo-bit request 

interfaces with alias addresses M, N. 0, and P are poked into vectors, so each poke group contains sixty-four bits. The 

pipe controller 24 3 . The poking times and busy-idle bit four poke groups can be labeled with a concatenation of the 

clearing times take place at different moments within each of 20 four alias labels on the request vectors. As a result, the four 

the pipe hunt controllers 24 0 -24 3 . From the point of view of poke groups for the re-designed pipe hunter of FIG. 5 are 

any pipe controller, the request vector bits flow through the called ABCD, EFGH, UKL, and MNOP. It is important to 

pipe controller in alphabetical order (A to P) if one ignores note that whenever a single sixty-four bit ABCD poke group 

the busy-idle bit clearing times. This ordering guarantees is being routed through one of the switch controllers in pipe 

that the aforementioned advantages of preferences will be 25 controller 24q of FIG. 5, there is also a sixty-four bit ABCD 

realized within the controller 20, because the request vector poke group being routed through each of the other fifteen 

generated from a input interface with alias address A will switch controllers in pipe controller 24o. As a result, there 

always be given precedence over the request vectors gen- are a total of 1024 request vector bits associated with sixteen 

erated from input Interfaces with alias addresses B, C and ABCD poke groups that are being routed through pipe Mo 

D, etc. 30 at a single instant of time. The modified controller 20 

The benefits derived from forced independence between processes the request vectors for all N input ports (by 

the inputs on a particular 16x16 crossbar switch produce a passing them through all four pipe hunt controllers 24 0 -24 3 ) 

slight increase in the complexity of the pipe hunter circuitry. every eight clock cycles, and since mis task must be com- 

Because of the independent connections between the input pleted within a single 176 nano seconds ATM cell interval, 

interfaces and the switch fabric 14 A, which independence is 35 the required clock rate within the controller 20 is 46 Mbps 

assured by the use of Galois field theory, a request vector regardless of Ihe size (aggregate throughput) of the NxN 

from a single input interface must be appropriately routed to ATM switch. As a result, since the controller 20 must 

several different switch controllers in each of the stages in perform eight processing steps (regardless of the network 

the pipe hunting ring. The mixing nature of the Galois field size), the process is said to be an 0(1) path hunt algorithm, 

theory generated connections requires each input interface 40 During the execution of this 0(1) path hunt algorithm for the 

120-12^3 to be connected to a different set of 16x16 N=256 input ATM switch 10A of FIG. 5, the equivalent of 

crossbar switches within the switch fabric 14A, and as a 16384 link controller path hunts and 16384 link controller 

consequence, it also requires request vectors generated on path hunt checks are performed every 176 uano seconds, so 

different input interfaces to be routed through entirely dif- if each path hunt is considered to be an instruction execution 

ferent sets of switch controllers within the controller 20. as and each path hunt check is considered to be an instruction 

Since request vectors are time- multiplexed on links within execution, then the controller 20 can be viewed as a parallel 

the controller 20. all of the request vectors (within a par- processor capable of sustaining a 186 gigabits per second 

ticular ATM cell slot) that are expelled from a particular processing rate. The trade-off for maintaining a reasonable 

switch controller in one pipe hunter stage must (by data rate in the controller 20 (regardless of size) is an 

definition) be routed to different switch controllers in the 50 increase in link controller logic complexity and an increase 

next pipe hunter stage. To provide this dynamic routing of in signal connections passing between successive stages of 

the request vectors, each pipe hunt controller 24 D , 24j J4 2 the controller as the size is increases. ATM switch designs 

and 24 3 is connected to a respective small switching network with aggregate throughputs in excess of 1 Terabits per 

30 0 , 30 lt 30 2 and 30 3 . shown in FIG. 5. Alternatively, simple second will require between 4096 and 32.768 signals (at 46 

multiplexers may be used instead of switching networks 30^ 55 Mbps) to be routed between successive pipe controller 

30^ 30 2 and 30 3 , thereby greatly decreasing costs for the stages. 

controller 20. Fortunately, the required configurations of In addition to increasing the number of signals between 

these small switching networks 30 o , 30 x . 30 2 and 30 3 .(or pipe controller stages, the use of parallelism within the 

multiplexers) are cyclic with a period equal to the ATM cell controller 20 also requires a slight increase in (he hardware 

period, and the required configurations can be determined a 60 requirements for each link controller, because each link 

priori and can therefore be 4 'hard- coded" into the small controller must now support a parallel path hunt on four bits 

switching networks (multiplexers) during the design of the within the poke group. Even though a few additions are 

circuitry of the controller 20. required to the hardware of the controller 20. the lower 

As mentioned previously, ATM switch 10A shown in FIG. processing rate resulting from the next level of parallelism 

5 might be scaled such that the number of input lines were 65 will typically justify minor increases in hardware. 

5 12. 1024 or even higher. For those size switches, assuming In a large ATM system, a large percentage of the overall 

that the input lines are carrying 2.5 Gigabits per second data system cost will undoubtedly be concentrated in the output 
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packet modules, because the functionality within the output switching fabric 14A in many different ways, likewise, the 

packet modules is relatively complex. In particular, each manner in which paths for arriving ATM cells are rapidly 

output packet module must provide a concentrator function' routed within the sections of the pipe hunter also provide 

that requires many FIFO-like memories, and the required opportunities far several features that would be very difficult 

access speed of these memories is directly proportional to 5 in an architecture based on in-band/self -routing control 

the number of inputs and outputs of the concentrator. Each techniques, as will be described below, 

packet module must also provide a large shared buffer The ATM switch shown in FIG. 5 may also have multiple 

memory in order to accommodate packets that are tempo- levels of cell loss priorities in order to provide different class 

rarily blocked from their desired output port and the of service levels to different customers. By defining a 

required access speed of this memory is proportional to the 10 specific bit within the ATM cell header as the cell loss 

number of outputs from the output packet module. The priority bit two different cell loss priority service classes can 

output packet module further provides a means of guiding be specified. Similarly. VFI/VCI field values or bits within 

packets to their desired output ports. Cell loss priorities and the user payload might be used to identify even more 

cell delay priorities must be managed within each output class-of-service priority levels. Using either of these 

packet module as well as within the line cards and distri- 15 techniques, the ATM switch can be operated such that a 

bution network. It should be evident that larger output higher priority cell always has a higher probability of being 

packet modules will make the task of providing all of these successfully routed through the network than a lower pri- 

functioDS even more expensive and complex, so tradeoff ority cell. Thus, if ATM cells are blocked and dropped, then 

techniques that help decrease the required size of the output the low priority cells should be the cells that are dropped, 

packet module are beneficial. 20 These cell loss priorities are completely different from the 

In the system shown in FIG. 5, each of the output packet preference weights described previously with respect to 

modules was designed to support sixty-four input links and switching network 14A. A high cell loss priority takes 

sixteen output links. This size is directly related to the precedence over a high preference weight (described 

number of crossbar switches used in the distribution previously), but two cells with the same priority level should 

network, because each output packet module must have 25 follow the arbitration rules defined by the ordering associ- 

exactly one link routed to it from each of the crossbar ated with the precedence weights. 

switches in the distribution network. The number of outputs The pipe hunter architecture has to be modified slightly 
emanating from the output packet module is always 1/F from its operation shown in FIG. 14, in order to provide 
times the number of inputs to the output packet module, multiple cell loss priorities. This is easily accomplished 
where F is the fanout provided by the distribution network. 30 because of the natural temporal ordering impressed on the 
The fanout F is four in the network shown in FIG. 5. request vectors as they pass through the pipe hunt control- 
As a result of these relationships, it should be evident that lers. This ordering puts an inherent level of priority to each 
decreasing the number of crossbar switches within the of the request vectors. Thus, the non-parallel approach, 
distribution network will result in smaller output packet shown in the timing diagram of FIG. 14, is modified to 
modules. However, to maintain low cell loss probabilities 35 permit two different priority level groups of request vectors 
within the network, larger crossbar switches will be required R* and R" to be passed through the pipe hunter in order to 
so that there will still be a sufficient number of alternate implement multiple priority levels. The request vectors 
paths through the distribution network. As an example, the associated with the high-priority ATM cells each is identified 
sixty-four 16x16 crossbar switches in FIG. 5 could be by a R\ and the request vectors associated with the low- 
replaced by thirty-two 32x32 crossbar switches, and the « priority ATM cells each is identified by a R". The timing of 
sixteen 64x16 output packet modules of FIG. 5 could be the higher priority group of request vectors and the lower 
replaced by thirty-two 32x8 output packet modules. Another priority group of request vectors is illustrated in FIG. 15. 
example of this approach occurs if the hardware of FIG. 5 In FIG. 15, the request vectors have subscripts ABCD, y; 
is replaced by sixteen 64x64 crossbar switches in the dis- EFGH. y, DKL. y ; and MNOP, y. The ABCD. EFGH, UKL, 
tribution network and sixty-four 16x4 output packet mod- 45 and MNOP subscripts represent request vectors arriving in 
ules. .These examples indicate the tradeoffs between hard- line cards with abases ABCD, EFGH, UKL, and MNOP 
ware in the distribution network and hardware in the output respectively. The subscript y represents the cell time slot 
packet modules that can be made within the design of the during which the ATM cells desire routing. In FIG. 15, these 
ATM switch architecture of the present invention. time slots are shown relative to an index time slot i. Along 
Unfortunately, these trade-offs do not come without cost 50 the left edge, each time slot is divided into twelve periods 
because the designs that have larger crossbar switches in me 0-11. The four columns to the right of the time column 
distribution network (and smaller output packet modules) represent pipe controller 240. pipe controller 241, pipe 
will typically produce lower overall system level costs, but controller 242 and pipe controller 243, respectively. The 
they also produce degraded levels of system performance in busy-idle bit vector for pipe p is for an ATM cell in time slot 
cell loss probabilities. This degradation in the cell loss 55 y is represented by Thus, for time sloti and pipe 0,Le., 
probability results from a decrease in the level of indepen- pipe 240, the busy-idle bit vector is represented by B 0J . The 
dence between inputs on the larger crossbar switches. shading indicates the specific time periods when request 
Nevertheless, many designs still produce acceptable levels vectors are poked into the pipe controllers. The lade of 
of cell loss without requiring the addition of a fifth active shading indicates that those request vectors atrived at that 
pipe in the distribution network. Simulations of the example 60 pipe controller via circulation around the circularly con- 
having thirty-two 32x32 crossbar switches and thirty-two nccted pipe controller described previously. For example, at 
32x8 output packet modules indicate a cell loss probability time i, 6 and i, 7 request vectors R' AB co.i k *abcdj arc 
of lxlO -12 . which is only slightly higher that the cell loss respectively poked into pipe controller 0 and logically 
probability for the network in FIG. 5. ANDED with B 0J and B 0J , sequentially. If the busy-idle bits 
The ATM switch architecture of the present invention 65 of vector B 0j indicate that a requested path is available in 
which is based on out-of-band control techniques permits pipe 0. then the path is assigned and the busy idle bit 
the use of global information with regard to the status of the representing that path and the request vector requesting that 
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path are changed to reflect that the path is committed to 
fulfill a request in the present time period. At time L 7 and 

1 8 these request vectors (changed to reflect any fulfilled 
requests) are routed to pipe controller 1. At time i, 8 and i, 

9 these request vectors (again changed to reflect any fulfilled 
requests) are routed to pipe controller 2. At time i, 9 and L 

10 these request vectors (again changed to reflect any 
fulfilled requests) are routed to pipe controller 3. At this time 
in pipe controller 3, request vectors R'xjcb.i and R">*cx>.i 
have the first two chances at selecting paths from a newly 
cleared pipe controller, the clearing of this pipe is repre- 
sented by the *3. Thus, vector R'^dj should find paths for 
all its unfulfilled requests unless they self-conflict. These 
request vectors would continue to circulate, but at time i, 11 
as these request vectors would return to pipe controller • the 
pipe controller is cleared (represented by *©) and the request 
vectors R\*ci>j and R'a^csu are discarded. As shown in 
FIG. 15, the other portions of the controller work in the same 
manner. 

It should be noted that the inclusion of cell loss priorities 
techniques does not (and should not) detract from the 
desirable attributes of rolling. In particular, each of the 
successive path hunt attempts for a particular ATM cell still 
occurs in a more lightly-loaded crossbar switch. 

If a non-parallel controller approach (as shown in FIG. 5) 
to process two priority level request vectors, a data rate 
speed-up to permit all of the request vectors to be routed 
through the controller within the 176 nsec ATM cell interval. 
If this is deemed undesirable, then a parallel approach that 
routes the high-priority request vectors and low-priority 
request vectors through the controller simultaneously could 
also be used, but more signals are required between con- 
secutive pipe controller stages and more complicated pro- 
cessing logic is required within the link controllers. 
Nevertheless, the required ordering needed to ensure that 
high-priority cells are routed before low-priority cells can 
still be maintained if the link controller hardware is designed 
with a hard-wired priority ordering. 

Analytical modeling has shown that the distribution net- 
work 14A works well with multiple cell loss priority levels. 
For example, if the fully-loaded network of FIG. 5 had four 
priority levels (highest priority=level 0 and lowest priority^ 
level 3) and has input traffic that is equally distributed across 
all four priority levels, then the distribution network 14A 
will route traffic of priority level 0 with an average cell loss *3 
probability of 10 . traffic of priority level 1 with an 
average cell loss probability of 10~ lg r traific of priority level 

2 with an average cell loss probability of 10~ 16 . and traffic 
of priority level 3 with an average cell loss probability of 
10~ 13 . Thus, it is quite apparent that the use of a high (and 50 
presumably more expensive) priority level can have a sig- 
nificant effect on the average cell loss probability that is 
experienced by a particular user. 

The ATM switch 10A may be realized with either elec- 
tronic or photonic/electronic components. Either embodi- 
ment may capitalize on the parallel processing architecture. 
An electronic ernJbodiment is the more attractive presently 
because of the inexpensive and ease of procuring the com- 
ponents necessary construction. However, a photonic/ 
electronic hybrid has great promise because of the possible 
advantages in connectivity. For a photonic/electronic 
embodiment* a novel interconnection technology known as 
free-space digital optics may be employed. System based on 
free-space digital optics use beams of light propagating 
roughly orthogonal to the plane of the device substrates to 
provide connections between devices. These beams are 
routed using lenses, mirrors, and holograms to the desired 
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receivers on the target device array. The free space optical 
approach to device connectivity may yield systems with 
very high degrees of connectivity between devices. 

Although the ATM packet switch 10A was initially 
designed with the parallelism of free-space digital optics in 
mind, one of the advantages of the its architecture results 
from the fact mat the switch fabric 14A can be implemented 
using any technology that can provide the functionality of 
many small, high-speed circuit switches. As a result, the 
architecture of switch fabric 14A is essentially technology- 
independent An embodiment using free-space digital optics 
as the interconnection technology within the switch fabric is 
contemplated. The 16x16 crossbar switches within the 
switch fabric 14A will be implemented with FET-SEED 
device arrays. This approach may provide many benefits 
within the switch fabric 14A, because the resulting design 
based on optical interconnections may have lower levels of 
signal crosstalk, lower chip counts (and potentially lower 
cost) due to increased device integration, lower signal skew, 
and lower overall power dissipation (resulting in simpler 
thermal management techniques within the switch fabric 
14 A). These benefits might also prove to be useful in a 
free-space optical implementation of the out-of-band con- 
troller 30, although the design of the 640 Gigabits per 
second controller 30 is contemplated to be based on elec- 
tronic technologies. 

While the invention has been particularly illustrated and 
described with reference to preferred embodiments thereof, 
it will be understood by those skilled in the art that various 
changes in form, details, and applications may be made 
therein. For example, four or more levels of priority may be 
implemented instead of the two shown and described. It is 
accordingly intended that the appended claims shall cover 
all such changes in form, details and applications which do 
not depart from the true spirit and scope of the invention. 

What is claimed is: 

1. A packet switch for switching a telecommunication 
packet from a plurality of input lines to a plurality of output 
lines, comprising: 

a plurality of input interfaces, each having an input port 
connected to a respective input line of said plurality of 
input lines, and each of said input interfaces having an 
output port; 

a network for switching a plurality of I input ports to a 

plurality of P output ports; 
each of said plurality of input interface output ports is 
fanned out to a respective group of F of said I input 
ports of said network, 
said network having a plurality of C pipes, where C is an 

integer of a value equal to P/I; 
a plurality of output modules, said output modules 
together having a plurality of inputs, each of said output 
module inputs connected to respective output port of 
said plurality of P output ports, and together having a 
plurality of outputs, each of said output module outputs 
connected to a respective output line of said plurality of 
output lines; 

each pipe of said C pipes having a path from each of the 
plurality of inputs lines that is conncctablc to a respec- 
tive output line of the plurality of output lines; and 
an out of band controller having multiple priority levels 
for hunting a path through said packet switch for a 
telecommu ni cation packet 

2. The packet switch as set forth in claim 1, wherein: 
each of said input interfaces has a store for storing a 

telecommunication packet; and 
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said out-of-band controller rolls a request for a path far 
the tclccommunicatioiis packet which was unable to 
find an unblocked path through a first pipe to an input 
of a pipe controller of a second pipe and the telecom- 
munications packet is stored in said input interface 
while me controller is hunting an unblocked path. 
3. The packet switch as set forth in claim 1, wherein: 
each of said input interfaces has a store for storing a 
telecommunication packet; and 



communication packet which was unable to find an 
unblocked path through a first pipe to an input of a second 
pipe. 

11. The packet switch as set forth in claim 9. wherein said 
out-of-band controller rolls a request for a path of a tele- 
communication packet which was unable to find an 
unblocked path through both a first pipe and a second pipe 
to an input of a pipe controller of a third pipe, 

12. The packet switch as set forth in claim 9. wherein said 



. . . , . . _ „ . - . frwr to out-of-band controller rolls a request for a path of a tele- 

said out-of-band controller rolls a request tor a path for . ^ ... . . ,. - . 

the ickammnnicaions packet which was unable to •^""^P"^ WM „ W - 5 «, 5 

tod an unblocked path through both . first pipe .ad a unhlockedpathtooughafir.lp^ ,,.«e^dpipe»nda.hn-d 
«»nd pipe to aaS^ut of »rt«coatrolleiTa third ppeto*. inpm of a ppe conn-oiler of a fourth ptpe. 

. , ■ ^ w-* „♦ i „m 13. The packet switch as set forth in claim 9, wherein said 

^^^TTT* ™^ Vt^STt « out-of-band control!* assigns an order cf preference to 
S J?^ ^ ^ ATM cells to reduce a rrobability of losing an ATM ceU by 

internal blocking. 

14. The packet switch as set forth in claim 9. wherein said 
out-of-band controller assigns an order of preference to 
20 ATM cells, and rolls a request for a path of a telecommu- 
nications packet which was unable to find an unblocked path 
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unblocked path. 

4. The packet switch as set forth in claim 1. wherein: 
each of said input interfaces has a store for storing a 

tdecommunication packet; and 
said out-of-band controller rolls a request far a path for 
the telecommunications packet which was unable to 
find an unblocked path through a first pipe, a second 
pipe and a third pipe to an input of a pipe controller of 
a fourth pipe and the telecommunication packet is 
stored in said input interface while the controller is 
hunting an unblocked path. 

5. The packet switch as set forth in claim 1, wherein said 
out-of-band controller assigns an order of preference to 
ATM cells to reduce a probability of losing an ATM cell by 
internal blocking. 

6. The packet switch as set forth in claim 1, wherein said 
out-of-band controller assigns an order of preference to 
ATM cells, and rolls a request for a path for a telecommu- 
nications packet which was unable to find an unblocked path 35 
through a first pipe to an input of a pipe controller of a 
second pipe. 

7. The packet switch as set forth in claim 1, wherein said 
out-of-band controller assigns an order of preference to 
ATM cells, and rolls a request for a path for a telecommu- 40 
nications packet which was unable to find an unblocked path 
through a first pipe and a second pipe to an input of a pipe 
controller of a third pipe. 

8. The packet switch as set forth in claim 1, wherein said 
out-of-band controller assigns an order of preference to 45 
ATM cells, and rolls a request for a path for a telecommu- 
nications packet which was unable to find an unblocked path 
through a first pipe, a second pipe and a third pipe to an input 
of a pipe controller of a fourth pipe. 

9. A packet switch for switching telecornmunication 
packets, comprising: 

a network for switching a plurality of I inputs to a 

plurality of P outputs; 
said network having a plurality of C pipes, where C is an 

integer of a value equal to Pft; 
each pipe having a respective pattern of switching its 

inputs to its outputs; 
each pattern of switching of a pipe is independent of the 

patterns of switching of the other pipes; 
a plurality of output modules connected to said plurality 

of P outputs; and 
an out of band controller having multiple priority levels 

for hunting a path through said packet switch for a 

telecommunication packet 
II. The packet switch as set forth in claim 9, wherein said 
out-of-band controller rolls a request for a path of a tclc- 
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through a first pipe to an input of a pipe controller of a 
second pipe. 

15. Hie packet switch as set forth in claim 9, wherein said 
out-of-band controller assigns an order of preference to 
ATM cells, and rolls a request for a path of a telecommu- 
nication packet which was unable to find an unblocked path 
through a first pipe and a second pipe to an input of a pipe 
controller of a third pipe. 

16. The packet switch as set forth in claim 9, wherein said 
out-of-band controller assigns an order of preference to 
ATM cells, and rolls a request for a path of a telecommu- 
nications packet which was unable to find an unblocked path 
through a first pipe, a second pipe and a third pipe to an input 
of a fourth pipe, 

17. The packet switch as set forth in claim 9, wherein said 
out-of-band controller controls finds and establishes paths 
through said packet switch for the tdecomrnunication pack- 
ets. 

18. An ATM switch for switching ATM packets, compris- 
ing: 

a plurality of ATM interface cards, each having a respec- 
tive input connected to an ATM telecommunication line 
and an output; 

a network for switching a plurality of I inputs to a 
plurality of P outputs; 

said network having a plurality of F pipes, where F is an 
integer of a value equal to P/I; 

each of said outputs of said input interfaces is fanned out 
by a factor of F and connected a respective input of 
each of said F pipes; 

each pipe having a respective pattern of switching its 
inputs to its outputs; 

each pattern of switching of a pipe is independent of the 
patterns of switching of the other pipes; 

a plurality of output modules connected to said plurality 
of P outputs; 

each of said output modules having a plurality of outputs; 
and 

an out of band controller having multiple priority levels 
for hunting a path from an input of one of said plurality 
of input interfaces through said ATM switch for an 
ATM packet to a desired output of one of said plurality 
of output modules. 

19. The ATM switch according to claim 18, wherein an 
input interface of said plurality of input interfaces has a 
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memory thai stores an ATM packet for two ATM packet 
periods to allow an ATM packet which was blocked during 
the previous ATM packet period to hunt a path from an input 
of one of said plurality of input interfaces through said ATM 
switch for an ATM packet to a desired output of one of said 5 
plurality of output modules during the next ATM packet 
period. 

20. The ATM switch according to claim 19. wherein said 
ATM packet which was previously blocked for one ATM 
period is communicated through said ATM switch to its 
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respective desired output of its output module in proper 
sequence without the use of a time stamp. 

21. The ATM switch according to claim 18, wherein an 
Input interface of said plurality of input interfaces has a 
means for reading a priority field of each incoming ATM 
packet and means for releasing a request vector during a pipe 
hunting period assigned to its ATM packets according to the 
value in said priority field. 
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